{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# BIDMach: parameter tuning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook we'll explore automated parameter exploration by grid search. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 CUDA device found, CUDA version 8.0\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(0.9898467,12669616128,12799574016)"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import BIDMat.{CMat,CSMat,DMat,Dict,IDict,FMat,FND,GDMat,GMat,GIMat,GSDMat,GSMat,HMat,Image,IMat,Mat,SMat,SBMat,SDMat}\n",
    "import BIDMat.MatFunctions._\n",
    "import BIDMat.SciFunctions._\n",
    "import BIDMat.Solvers._\n",
    "import BIDMat.JPlotting._\n",
    "import BIDMach.Learner\n",
    "import BIDMach.models.{FM,GLM,KMeans,KMeansw,ICA,LDA,LDAgibbs,NMF,RandomForest,SFA}\n",
    "import BIDMach.datasources.{MatSource,FileSource,SFileSource}\n",
    "import BIDMach.mixins.{CosineSim,Perplexity,Top,L1Regularizer,L2Regularizer}\n",
    "import BIDMach.updaters.{ADAGrad,Batch,BatchNorm,IncMult,IncNorm,Telescoping}\n",
    "import BIDMach.causal.{IPTW}\n",
    "\n",
    "Mat.checkMKL\n",
    "Mat.checkCUDA\n",
    "Mat.setInline\n",
    "if (Mat.hasCUDA > 0) GPUmem"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset: Reuters RCV1 V2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The dataset is the widely used Reuters news article dataset RCV1 V2. This dataset and several others are loaded by running the script <code>getdata.sh</code> from the BIDMach/scripts directory. The data include both train and test subsets, and train and test labels (cats). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "data": {
      "text/plain": [
       "4.284"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "var dir = \"../data/rcv1/\"             // adjust to point to the BIDMach/data/rcv1 directory\n",
    "tic\n",
    "val train = loadSMat(dir+\"docs.smat.lz4\")\n",
    "val cats = loadFMat(dir+\"cats.fmat.lz4\")\n",
    "val test = loadSMat(dir+\"testdocs.smat.lz4\")\n",
    "val tcats = loadFMat(dir+\"testcats.fmat.lz4\")\n",
    "toc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First lets enumerate some parameter combinations for learning rate and time exponent of the optimizer (texp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "data": {
      "text/plain": [
       "  0.30000\n",
       "  0.40000\n",
       "  0.50000\n",
       "  0.60000\n",
       "  0.70000\n"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "val lrates = col(0.03f, 0.1f, 0.3f, 1f)        // 4 values\n",
    "val texps = col(0.3f, 0.4f, 0.5f, 0.6f, 0.7f)  // 5 values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next step is to enumerate all pairs of parameters. We can do this using the kron operator for now, this will eventually be a custom function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "data": {
      "text/plain": [
       "  0.030000   0.30000\n",
       "   0.10000   0.30000\n",
       "   0.30000   0.30000\n",
       "         1   0.30000\n",
       "  0.030000   0.40000\n",
       "   0.10000   0.40000\n",
       "   0.30000   0.40000\n",
       "         1   0.40000\n",
       "        ..        ..\n"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "val lrateparams = ones(texps.nrows, 1) ⊗ lrates\n",
    "val texpparams = texps ⊗ ones(lrates.nrows,1)\n",
    "lrateparams \\ texpparams"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's the learner again:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "data": {
      "text/plain": [
       "BIDMach.models.GLM$LearnOptions@3408ab31"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "val (mm, opts) = GLM.learner(train, cats, GLM.logistic)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To keep things simple, we'll focus on just one category and train many models for it. The \"targmap\" option specifies a mapping from the actual base categories to the model categories. We'll map from category six to all our models:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "data": {
      "text/plain": [
       "   0   0   0   0   0   0   1   0   0   0   0   0   0   0   0   0   0   0...\n",
       "   0   0   0   0   0   0   1   0   0   0   0   0   0   0   0   0   0   0...\n",
       "   0   0   0   0   0   0   1   0   0   0   0   0   0   0   0   0   0   0...\n",
       "   0   0   0   0   0   0   1   0   0   0   0   0   0   0   0   0   0   0...\n",
       "   0   0   0   0   0   0   1   0   0   0   0   0   0   0   0   0   0   0...\n",
       "   0   0   0   0   0   0   1   0   0   0   0   0   0   0   0   0   0   0...\n",
       "   0   0   0   0   0   0   1   0   0   0   0   0   0   0   0   0   0   0...\n",
       "   0   0   0   0   0   0   1   0   0   0   0   0   0   0   0   0   0   0...\n",
       "  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..  ..\n"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "val nparams = lrateparams.length\n",
    "val targmap = zeros(nparams, 103)\n",
    "targmap(?,6) = 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "data": {
      "text/plain": [
       "  0.30000\n",
       "  0.30000\n",
       "  0.30000\n",
       "  0.30000\n",
       "  0.40000\n",
       "  0.40000\n",
       "  0.40000\n",
       "  0.40000\n",
       "       ..\n"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "opts.targmap = targmap\n",
    "opts.lrate = lrateparams\n",
    "opts.texp = texpparams"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "corpus perplexity=5582.125391\n",
      "pass= 0\n",
      " 2.00%, ll=-0.69315, gf=0.184, secs=1.5, GB=0.02, MB/s=13.94, GPUmem=0.971496\n",
      "16.00%, ll=-0.40496, gf=1.232, secs=1.6, GB=0.13, MB/s=82.82, GPUmem=0.971496\n",
      "30.00%, ll=-0.37984, gf=2.131, secs=1.7, GB=0.25, MB/s=141.66, GPUmem=0.971496\n",
      "44.00%, ll=-0.31457, gf=2.894, secs=1.9, GB=0.36, MB/s=191.87, GPUmem=0.971496\n",
      "58.00%, ll=-0.34047, gf=2.904, secs=2.5, GB=0.48, MB/s=192.35, GPUmem=0.971496\n",
      "72.00%, ll=-0.23351, gf=3.437, secs=2.6, GB=0.59, MB/s=227.23, GPUmem=0.971496\n",
      "87.00%, ll=-0.28136, gf=3.948, secs=2.7, GB=0.70, MB/s=260.96, GPUmem=0.971496\n",
      "100.00%, ll=-0.23004, gf=4.386, secs=2.8, GB=0.81, MB/s=288.84, GPUmem=0.971250\n",
      "pass= 1\n",
      " 2.00%, ll=-0.28089, gf=4.439, secs=2.8, GB=0.83, MB/s=293.34, GPUmem=0.971250\n",
      "16.00%, ll=-0.22612, gf=4.867, secs=2.9, GB=0.94, MB/s=321.43, GPUmem=0.971250\n",
      "30.00%, ll=-0.28134, gf=5.249, secs=3.0, GB=1.05, MB/s=346.45, GPUmem=0.971250\n",
      "44.00%, ll=-0.27640, gf=5.616, secs=3.1, GB=1.17, MB/s=370.62, GPUmem=0.971250\n",
      "58.00%, ll=-0.23583, gf=5.971, secs=3.3, GB=1.28, MB/s=394.11, GPUmem=0.971250\n",
      "72.00%, ll=-0.19559, gf=6.207, secs=3.4, GB=1.39, MB/s=409.45, GPUmem=0.971250\n",
      "87.00%, ll=-0.27391, gf=6.503, secs=3.5, GB=1.51, MB/s=428.98, GPUmem=0.971250\n",
      "100.00%, ll=-0.21918, gf=6.754, secs=3.6, GB=1.61, MB/s=444.78, GPUmem=0.971250\n",
      "Time=3.6260 secs, gflops=6.75\n"
     ]
    }
   ],
   "source": [
    "mm.train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "data": {
      "text/plain": [
       "BIDMach.models.GLM$PredOptions@3bbdc0b8"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "val (pp, popts) = GLM.predictor(mm.model, test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And invoke the predict method on the predictor:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "corpus perplexity=65579.335560\n",
      "Predicting\n",
      " 3.00%, ll=0.00000, gf=0.013, secs=0.3, GB=0.00, MB/s= 1.63, GPUmem=0.98\n",
      " 6.00%, ll=0.00000, gf=0.025, secs=0.3, GB=0.00, MB/s= 3.06, GPUmem=0.98\n",
      "10.00%, ll=0.00000, gf=0.037, secs=0.3, GB=0.00, MB/s= 4.32, GPUmem=0.98\n",
      "13.00%, ll=0.00000, gf=0.050, secs=0.3, GB=0.00, MB/s= 5.90, GPUmem=0.98\n",
      "16.00%, ll=0.00000, gf=0.062, secs=0.3, GB=0.00, MB/s= 7.30, GPUmem=0.98\n",
      "20.00%, ll=0.00000, gf=0.073, secs=0.3, GB=0.00, MB/s= 8.60, GPUmem=0.98\n",
      "23.00%, ll=0.00000, gf=0.084, secs=0.3, GB=0.00, MB/s= 9.86, GPUmem=0.98\n",
      "26.00%, ll=0.00000, gf=0.095, secs=0.3, GB=0.00, MB/s=11.17, GPUmem=0.98\n",
      "30.00%, ll=0.00000, gf=0.107, secs=0.3, GB=0.00, MB/s=12.52, GPUmem=0.98\n",
      "33.00%, ll=0.00000, gf=0.118, secs=0.3, GB=0.00, MB/s=13.88, GPUmem=0.98\n",
      "36.00%, ll=0.00000, gf=0.129, secs=0.3, GB=0.01, MB/s=15.21, GPUmem=0.98\n",
      "40.00%, ll=0.00000, gf=0.140, secs=0.3, GB=0.01, MB/s=16.47, GPUmem=0.98\n",
      "43.00%, ll=0.00000, gf=0.151, secs=0.3, GB=0.01, MB/s=17.72, GPUmem=0.98\n",
      "46.00%, ll=0.00000, gf=0.161, secs=0.3, GB=0.01, MB/s=19.19, GPUmem=0.98\n",
      "50.00%, ll=0.00000, gf=0.172, secs=0.3, GB=0.01, MB/s=20.46, GPUmem=0.98\n",
      "53.00%, ll=0.00000, gf=0.182, secs=0.3, GB=0.01, MB/s=21.61, GPUmem=0.98\n",
      "56.00%, ll=0.00000, gf=0.193, secs=0.3, GB=0.01, MB/s=23.01, GPUmem=0.98\n",
      "60.00%, ll=0.00000, gf=0.204, secs=0.3, GB=0.01, MB/s=24.27, GPUmem=0.98\n",
      "63.00%, ll=0.00000, gf=0.214, secs=0.3, GB=0.01, MB/s=25.36, GPUmem=0.98\n",
      "66.00%, ll=0.00000, gf=0.224, secs=0.3, GB=0.01, MB/s=26.67, GPUmem=0.98\n",
      "70.00%, ll=0.00000, gf=0.234, secs=0.4, GB=0.01, MB/s=27.81, GPUmem=0.98\n",
      "73.00%, ll=0.00000, gf=0.245, secs=0.4, GB=0.01, MB/s=29.10, GPUmem=0.98\n",
      "76.00%, ll=0.00000, gf=0.254, secs=0.4, GB=0.01, MB/s=30.13, GPUmem=0.98\n",
      "80.00%, ll=0.00000, gf=0.265, secs=0.4, GB=0.01, MB/s=31.52, GPUmem=0.98\n",
      "83.00%, ll=0.00000, gf=0.275, secs=0.4, GB=0.01, MB/s=32.70, GPUmem=0.98\n",
      "86.00%, ll=0.00000, gf=0.285, secs=0.4, GB=0.01, MB/s=33.81, GPUmem=0.98\n",
      "90.00%, ll=0.00000, gf=0.294, secs=0.4, GB=0.01, MB/s=34.91, GPUmem=0.98\n",
      "93.00%, ll=0.00000, gf=0.305, secs=0.4, GB=0.01, MB/s=36.27, GPUmem=0.98\n",
      "96.00%, ll=0.00000, gf=0.315, secs=0.4, GB=0.01, MB/s=37.42, GPUmem=0.98\n",
      "100.00%, ll=0.00000, gf=0.323, secs=0.4, GB=0.01, MB/s=38.50, GPUmem=0.98\n",
      "Time=0.3660 secs, gflops=0.32\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "  0.00020220     0.99998     0.99999     0.99999     0.76872  0.00029059...\n",
       "  3.0089e-05      1.0000      1.0000      1.0000     0.89846  5.0510e-05...\n",
       "  5.3052e-07           1           1           1     0.98357  2.0036e-05...\n",
       "  1.4812e-23           1           1           1           1  5.7669e-20...\n",
       "  0.00042768     0.99997     0.99999     0.99999     0.74460  0.00053930...\n",
       "  6.2601e-05      1.0000      1.0000      1.0000     0.84036  0.00010468...\n",
       "  7.3177e-06           1      1.0000      1.0000     0.95748  9.3609e-06...\n",
       "  5.8293e-18           1           1           1     0.99985  5.1645e-15...\n",
       "          ..          ..          ..          ..          ..          ..\n"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pp.predict\n",
    "val preds = FMat(pp.preds(0))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pp.model.asInstanceOf[GLM].mats.length"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Although ll values are printed above, they are not meaningful (there is no target to compare the prediction with). \n",
    "\n",
    "We can now compare the accuracy of predictions (preds matrix) with ground truth (the tcats matrix). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "data": {
      "text/plain": [
       "-0.23961"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "val vcats = targmap * tcats                                          // create some virtual cats\n",
    "val lls = mean(ln(1e-7f + vcats ∘ preds + (1-vcats) ∘ (1-preds)),2)  // actual logistic likelihood\n",
    "mean(lls)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A more thorough measure is ROC area:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "data": {
      "text/plain": [
       "        0        0        0        0        0        0        0        0...\n",
       "  0.84452  0.82876  0.67050  0.72594  0.83775  0.83942  0.76525  0.72817...\n",
       "  0.88921  0.88569  0.79612  0.83757  0.88726  0.88837  0.84971  0.82579...\n",
       "  0.91906  0.91396  0.86260  0.88587  0.91758  0.91897  0.89496  0.88708...\n",
       "  0.93538  0.93176  0.90247  0.91155  0.93334  0.93501  0.92101  0.91433...\n",
       "  0.94604  0.94178  0.92518  0.92620  0.94474  0.94502  0.93492  0.93232...\n",
       "  0.95299  0.95031  0.94085  0.93575  0.95216  0.95272  0.94539  0.94215...\n",
       "  0.95782  0.95624  0.95095  0.94345  0.95744  0.95791  0.95290  0.95012...\n",
       "       ..       ..       ..       ..       ..       ..       ..       ..\n"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "val rocs = roc2(preds, vcats, 1-vcats, 100)   // Compute ROC curves for all categories"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfQAAAGQCAYAAABYs5LGAABJnUlEQVR42u29CVBUaZ722xETE3Mn4nbMvXEjJvqLmJjbMd+dmJju6e7p7pnq6p6evn42Xqp17K7WymRHFEEEQRApcGMVBEFFLHBFRVARi8KlEMtdURFFlnIrF3BDRFERFRRZnnv+x0ompZIlWTLPIZ+f8QZ5DiflOU9mnuf9v+85eb4HQgghhOie79ECQgghhIFOCCGEEAY6IYQQQhjohBBCCGGgEzKkD833vjeoNtDzHJnr168jPj4eP/vZz+gLIQx0QhjoY9FDQggDnRC7hNFoP28sIpX50qVL1UqdvhDCQCdEV4FO6CchDHRCHKBCT0pKwg9/+EP85V/+Jf7xH/8RqampAz5vpH73+vVrzJ07F3/zN3+Dv/qrv3pv25s3b6q/++d//mf89V//tarvBz/4ATw9PXH+/HkGOiEMdEIY6CYMBoPF+eSgoCCbBLqrq2vP47/4i7/o2S4/P18N+P7mvOXkNgY6IQx0QnQf6MM9Ka6wsPC99b/85S/x+PFjtX3wwQc2CfR/+qd/wpUrV97bpqqq6r1tnJyc0NjYiI6ODvj7+7/3u7179zLQCWGgE+LYgT5p0qT31h8+fLjnd/J4sMHce6jcmkD/6quvvqPXvGqXVltb2/O79vb29343btw4BjohDHRC9B3ow33e97///ffWd3V19RmcoxXo5n/ThMynD7bzInPrDHRCGOiEOHSgy0lmQ71+faQC3RK9dVk7IsFAJ4SBTggr9CFU6OYns8n/MdxA761LtGjJT0IIA50QTQX6n/70p/fWHzt2rOd3A82h966iTaErl5MNN9B7n3mfl5fHQCeEgU4IA72v5+3fv/+99b/97W97znL/8MMP+/17P//5z9/73aZNm9Rryp2dnYcd6L07BXLt+YkTJ3pGAC5duoTMzExVIwOdEAY6IQ4f6IK3t/eQrkPfvHmzxefJF8EMN9BN/78M5Y/0/Ploz8sTwkAnhNgl0AX5Zrh/+Id/UIfR5eeKFSvUa74HOpt8zZo1Pc/7+7//e/V5A/09a/ZBrk+X687lWnX5GxLwMr8uowCxsbGoqalhoBPCQCeE9EdBQcF7ASchSgghDHRCNMrvfvc77NixA0+ePFGX5eQ2Wf7bv/3bAb8AhhBCGOiEaOWDOMDQs1xjvn79ehpFCGGgE6JlKioq1BPZfvKTn6jz5BLgMk89fvx4LF68GHfu3KFJhBAGOiGEEMJAJ4QQQggDnRBCCCEMdEIIIYQw0AkhhBAGOiGEEEIY6IQQQghhoBNCCCGEgU4IIYQw0AkhhBDCQNcqVVVV7y37+fkhJiZGs02+CpT66CE10kN66JgehoaGMtD74sSJE+8tT5s2TdN6jx8/Tn30kBrpIT10UA8l1Bnogwx06VFpmdu3b1MfPaRGekgPHdRDBroVgW6tWYQQQoitYKCzQqc+ekgPqZEeskJ3rEDnHPrY1kcP6SE10kM962Ogs0KnPnpID6mRHrJCtx/jxo3raYMlOzsbH330kdo2btxodaBzDp0QQohW0X2FPthALy4uRkhICFpaWtQm1+vJOlbo1EcP6SE10kNW6DoKdAnz6urqnmV5HBwcbFWgcw59bOujh/SQGumhnvU5TKA7Ozuju7u7Z1keyzpW6NRHD+khNdJDVug6CnRL2zk5OVkV6JxDJ4RoiW7l31vlX7Pyr03596q7BS2dT3Dj+Tk8bL6Ouw+qUXunHCXnt6Li0iGcrj6AIxVFyPwqCwUntiP/6HbkHsxFSv5mrNm1CVkF2cjI24SUDRuQvjkLa7MzsWHjGmxbsxKb1yotS2mZK7AjfcW7dVnvWu6adORmpSMncyW2KW13ehoKVq/EjjUrkK9stzc1FXtXKOvSV2H3qnTsX7YS+5NW4vO0dLXtT1qBLxOV5dR3y/uU3xcnrkCh8viLtFXYv3wFDi5NQdFKZXmF8vuUVSiJS1UeZ6BQaXuSV6NkiSyvUdpq7Fu+CkcWJSt/dzX2pqTjwNKVOBWeiH1Jyu8SV2N/wmqcnJuEfcrPfUsz8GXcSpwNjkfx0nS1HUhMR1mQspywCsXxq1Ecl47zAbE4EKP8LnolShavQKXfYny1MBUHF6ep7aLfEhxakPJuWdFywT8ah6KS1cfSygOicThymfJ4eU/bnrmWgT7aFbp8h7uEeX5+/nvrfX191WESU89KfmppuaysjPqGuVxUVER9w1yW17ln+dgx3L15E93t7bhx4woOH/kSFVeP48nTBtRcUIKmsAilGz/H1VMXcaygBLtW70RR1DYc3HhUOVDvw864XSgJ24aixM+xK2EndsYqv4vKQmHMVuyKzkH+4s34KmwdCqM2Y9fCzShQ2qGItdgXtQF7FmxSf5bNScPxsM/w1fy1ODQvSzkwL8OZ4DQcC1+N4+HpuBgQh/I5S3FqXipKw5ajxn8JqgJiUDo3RWnLUT0zWm2lwctxak4qKmfEotonBicCV+C40sqnL0W1dwyOzU5T2/kZ8bjstRDHA5bjmNLOzUjAFc8FODErBSf9UnDBJw61ruE4PSMRZ5XnVnlF44FLMCqmxamPr3kswDPDTFx3j0StWzgeGIPRbvBAg8scNBoD8cglEB0GNzx2mY0nxllq6zK44LnRFy8N09Fm8AYMBrwxePa0boNReY47Og2u6mP5vfzsNLr2rHtrdMdroydajd7qupcuPnjq6odHrrOVddOUvxeAetc5antl8EGTyyw8cJujtCC8NCrLrrNwz2Mu7ijtqdEPD10DcdNzPq57RSiPg3DfNQRXvaNwZVoU7rmHKG0uLk9foLa77qGodwvuWb7tEab+319PX6i0RbjlEa7+vkZ5LO2613w0KP9nzYyFqPFdiOveyt9wC0S18rjKdxEuT4tEg/L8CiVkzyvBWum7GPeV55+bHYPy2dGomLUY9e5zcC4oRgnuaJwLjFH+/zk4GxT7bYtR9+tMcJzSYnEmJFbZt2CUhsYp75MEnAxPwE2vcJxUHh+fn6i2697zcSo0ASeUx9K+mRaB03Pj3y2HJ+GqT6Tyf8Wrj00tR+ncjOTnz6Hn0GWdNRU659DHtr7R1tjR2Y17T17j7tM3uPH4FWoePMf2c7ewt7oWxTW3UHCuEsnbi7Dmiy+Rt+8Q8r44gC2rtigVzzbsyN6OnWtzsE/p/e+LzcCuxI3YFb8RBxZkojhqLXYv2YzdizbjYEQWSpRWGLURX0RuwOGwdBwLScOX8zKVYFyD04FKoPgn4FjwKpxQWoWfHGgWKge5ZagISMA3Mz5Fg0cgrs2MxPUZEbjnFYznrjNw1zsE973m4LHHLLQpB/3HHv546j4TLW4+aii8dJ2GVldvvDF6vAsP47uQMIWHhMVbJVDeKM+VZQmNF0r4NBtnquHRZJyNRsMcNHwbXg9cQnDXJQy3Xeeh1TBN+RmOG66f4hu3KDxXwu6GWySueCzEZc+FaqBd94hAjbdycJ+2CI1K4F3xWoBy31ic9ktQfheJciWEjwYl4khwEqqVbU77x+PgPKVamp+M8plxODl7qeJjCvYvXK4E9VJ1u30xynLsMhxVDtrHwuOVCnIpDixTHkdF4+Sni3EwNRZfpcbgZPQinI2MwtG0WBxT1pXGKQGxYBFOLI9TWjxOx8XgfKQS6ClKECjLZxJjUbFkEUqXJ6I0JQllSYmojlY6EMlJOJWyXFmfjEr5OytTcWR1Kg4rVe6Z9Uk4qFS3JSvX4Ku0NTi/YiUOLduAkmWbcSBps1KNrlH07MLRjL0ozT6EEytyUL39DK7sOIebuypRnbgftXuv4v7JWjRW3sO9s9/gyY3HeH7vGZ7XP0fjtQa8fPQKb16+Refbbn6WdapvzAZ67/Wms9ybm5t5lvsY1NelHIPuPW5HWXUTam49wfmrD/DV6evYVlCOwqM12LrvHFbnHMXy5UVYnfEl1qTswfqFBdgXlI3syJ3YGrUDuRG5SkW1CjsitqAgfBOKwtehyicaxaEZ+Cp0FY7OTcVdrxCcDU5AudKLvxi4BE+UULvqH4HrM+ehbnqIGnYPvfzR6OWHJ54zlKrIFa9cvdDq5oXXLu/Crsvo8n6VpGzT/m11JI9fG73Uauup0V/5/6bhqcEfdUYlzAyL8cAQglvGSFS5JaDKPR73jGGoc52PSs9YVHrHok6p6G65hythFo+zfvH42nsRLilBdyx4GQ6HJuO0EtrnZsa+C69FKUp1m4TjSph9npSKgpQ07I5NR+H8DOzMXIn8DSuxI3M18hKyULBzBXbtWoGduelKJ2M9du9Jxa59K7B9z2qszc7GroMpKDiUjPzDKdi0PxO7T8Xj87Ox+OLCIhR8HY19FZHYfzECBy9GYf/1MByticKpS1E4e3kBTt6Yjerri3GlNgE3apej9l4IGh6sRXNTHlqfF+LNi3S8bSvB2/aT6Og4g7dvP1d+limtSmlfo739CLq6apR3wU2l3VJaudJuKO2h0p4orUlpreoAtR4+J47+WaaHDhjo5tehW7oe3VLQy7XnvA7dtjzv7ETty5c4fus2tm45jC1r92HDls+R9VkediiVZuGSVciLz0RhdDpKQ2NxYc5iHFOCp3ROEi7PWIRbXuGomhmDSzMW47ZnKB67zUKtdzhue4XhkftsvFIqvWZXXyUsvd6FpcGlZ9hQKjsJ1BaDr1LV+aLFOEOpEt3UobQ6j1Dc9JqHpy5+6hDdxZmLlbYEja6zcWZ2Ak7NSVQq2GTc8pyHr8JX4MD8Vdj/6RqU+8SjaIFSAUcroRarVMiBnynVcjZ2rtiMHWuUanpxLnasVzoEW7LxRd4mbFtZgL27NuHgvjU4emglioqycfJkGi5UxqDm0mKcuZCMazcWoP7+XDx6FIj6+lDl52w0PZqJRw988fiRPx41+OLR3el4rLTmJjc8qfdG0405ePpNGFpq/fHqRjDariYrbRXarsQqPxPw9voWdNzaju6ba4HadcD9Q0q+lSpNqQoeK+/r13VA58Nvw+6Z0t72BB4hRJ/wm+JYoVtETjdoaGjHpUtPUVJyA3mbTqFoYTryl67G9uR07I5LxsXg+TgdthAnw5egbO5C3PNWgtInGLUzgvHAe5YSrB54K3N0SkX60mUaXhqno8kQgBuGBbjsGq1WltfdlErNJw1HfDNweloqSpVWNHcTdkdsxt6wLOybm4Hc+C3ISczB9vhNSidgA3JXb0LeZxuwc8067FqzHvlbMrArLw1f7EvGgWMJ+PJgBA4dm4sz5wNx/bY/rlR548YlT9y95Y4XLa54eMcdTQ/c0PLEDZ0dRrx+4apUgC7ofOui7LkBXR0uePvKA+3N09D11og3DwPRXj8X7fci0NHsrfxUAvPBCrx9lInOloXoat2J7s79ynOV0ES20k4q7YLSpGo8qrQr31aOd5Qm0z53vw3S59+GaZsMyA8qUFkZsbqkPnrIQB+BQNf7HPrDp29RdecFcs/eQcruUqxPXIuMlI3YEbMaX326DA1eStjODMU9nyA885zx7kQZFzelKvbDLfd5eGiYg0rDUhxxy8IB7w0455GCgzPW4/OgndgVuht7AnKxM7QIOQsPYvPio9jw6TmsW1yONcv3ICNtAzalZCstHRvTo7EpcwkK8yOwr2gu9hbMxuF9SqVcPh3XL3nj+oVpuHt1Gpofu6HtlQva21zR3WVEZ7sLOl674k3zDCVsZ6GjxQ+drT7oerkQaI1Xqs44pdCcr7QNSj7mKhVonpKRnyl7fkRpSnWKM0qT4L2qtFql3VPadaU1K+2V0t4o1fNRXb/G1EgPqc9xPWSgj6EKvVbp8TV2AXkVjYhafQoH/ddhe1guimdl4bT3cvUM2CbDbPUEJTlDtcXoiwqveByemYGiwM0onr4e2+YVYdOiA8iK+wpLYyqwOukgNiyTIF6N3OVbsCk1CSuTFiE+dgmSk5dg7WcRyM8OwsECf1yq9MaV80ogn/XFvUo/NNd74kWDD14/CUBH60ylinVVAtYIdHkpYTtb+RmgtFBF+UqlrVPaaqVlKu2w0s4p7bzSLimtBbYaDmZlRA+pkR6yQneAQNfSHHpHRzdKjlYja+tJbIjaj0MB2XhqnIWH354lfM8zUK2sT4bH4vjC5Ti7aA2uf6q0iGyc/zQTX0YnYF1cMhLiFmBp/EKsTAzG8qS5yE7wwd5Ub5RumI5b5z1Qe8QP90+G4nFlONqapuH101noejNLCWJPRYUSzpiptEVKS1Pa0m+DeS/eDT9//m11LFXwA7w7QakDhBBCRh4Guk4q9H2nvsH62TuRP2sHLnivRoNruDokLgF+fmY0ipYkoGRRND5fH4fPMxYgJyEEcTGLlUp6MRJjlcCODkdKbCS2x4XgwMp5OJOzCHcvzEJ9ZQpePExFx5t45a94K81DaX5Ki/42qGVOeJ/Sjiltj9IuKq0B784aZq+e+ughPaRGVug6DHRbzqHvPXMVJyNycG7mWjW83xi88cLoi5MBq7Bl2Qos+Twec0tSsSJ3FtJSgpRKezHilJYVNxsFSXNQsmoh7l6cjcvnUvHoYTza2+Wae4PSpistQmnJSotS2na8O3GrUmkVSnsxavvEeTd6SH30kBpHTx8DXSMV+q36NiRH78OR0C1o8lyAdiXAZb57n+dubEzdhVn35sDjrRsWHQ3Ego1KtZ00D5+tCca+PA+cKg7F5eplaGubhs5Oaf7o7pbh74VKO/BtWNfj3dnU7DGzV08P6SE9ZIXOOfQR/xuP65+jMDIXrW6BSoh7ocSYg0y3M0hf8RVm1UXBu80doRcCsWKHH86f98D1yj/g0R0PvHkzA+/msCW0130b3NJaQAghxPFgoNupQv/mqxrcnr4EHS4++MYlHUsNNYj0+QYVeTuReiMcXi89MLfCC6dLP8DjO87qNdId7f54d6KZnP3dqLQu9urZq6dGekh99JCBbm2gj8QceuvLDhRE5qDD6IMbxiTM+aQR8/zX417eLuRcjoZnqzuiq/6Mm3f+A11dBqV9indnjL8a8P/mvNvY10gP6SH10UMGup0r9Pb2LsQuLUXD9Ag0usQh3HAPAX88ifK0LXi2fTdCav3g8dqAiof/ia5OV3SrX4hSzR4pNdJDekh99JCBPtKBPtQ59K4uoDjkM3QaPbDNeAQRE06iYP5SdH5ehNJ16zDvugfi3/wJ7d2fKEGeg3ffXEYIIYQw0DVToUuYZ8/PwXNjEBYbbiH9Jx/jfFgmXuzcjcKsZCx87orVXZ+g5ZUMrT9jj5Qa6SE9pIf0kIE+2oFu7Ry63OAk/9MteOESgqhPbiHv//4HPEjYj4bNu/FlSRCCug347PVkND/fNSJ6Oe9GD6mRHlKf43rIQB+lCv3Nmy7s+3QTXrqEInzKbSTOcMbLjV/i5ppcHNw+F3HdRnz6ehJevqxkj5Qa6SH10UN66NiBnp2dPej7m7948QIJCQmYMGECJk+ejJycHKsD3RqzKkNW461hOkKn3IXzwn9F3epiPF6/FSdL5mB/lyfmtk7Ek+bDIIQQQhw60IuLixESEoKWlha1hYaGquv6IikpCampqejo6EB7ezvS0tJw4MCBUanQ79xtRpuLH+INl/HT9F8hL3IfXuzYiZpKf9zoDIBPxxRcfpzJHik10kPqo4f0kIEuYV5d/d+Xdcnj4ODgPreXylyC3IQ8DgsLsyrQBzuHfmruWpw1ZuL3kVEIXLgGL3P34/bXs9Dy1gf+7Z/gi4YZo+IJ593oITXSQ+pzXA91G+jOzs7o7v7ve2TLY1k32ECXSr2/7YdaoRcf/wZvjDOw0HAO/8/Wn6Ii5SIe7UjF6zZ3LGvzw7KmPyha29kjpUZ6SH30kB4y0IVx48Z9Z52Tk1Of2ycmJqrD7BLqpiH3/ra3FOiDMeubmcn4wliAH27+KaKDr6NrRyEaG6Yh+9UseHZ8jJdv7oAQQghhoA+xQpeT4uLj49VKfcqUKdizZw8mTZpkcduqqio1zPPz899b7+vrqw6TmHpW8tN8efnqErwwBiDKUITfJ85D57YDuHE0BE8bgxD+cjK23Qvs9/nDXS4rKxvV/3+s65OfRUVF1DfMZXmdqW94y6ZGfTweWrM8pubQZd1gOXjwoHqinDUVen9z6Feev0KjVxSyjUfwf+z5AaoyyvB23ya8bnNDTfMazHj7Z3SiY1Q94bwbPaRGekh9juuh7s9yb25utniWe+8h+fT0dHVbqeTLy8vh6emJ+vp6qwK9vzn0r4Iy0GwIQIhHKn60LBwdW3bj8f3paH6Wh6WPxiPv9fJR94TzbvSQGukh9Tmuh7q+Dl2uPe/rOvTegS4VuQy1y5B7eHg46urqBvz/rZlDr3OLQYFxH/73kv8TO5bdQcf1BWh96Y5bDSvg1fkJniv/CCGEkNGC3xQ3AhX6sbJbaDPOwAL3XPxTahQ60irx4o4/HtzegM9u/wIrOuLYI2Wvnh5SHz2kRlboWgn0vubQ983bhKOGDfjBrr/DlrVNwBe78Oa1K67fWIiAdhdcV/7ZAs670UNqpIfU57geMtBHoEJvdpuDla4X8YNtv0XH6od4W5aAm9f8sf3y/0REdxh7pOzV00Pqo4fUyApdS4FuyayS/HN4ZgxFmMs2eKzZA6w8jdZ703H2TBAWtkzGMeUfIYQQMtow0IdZoZcGrMAXxiL8dOUvcGzNQ6B4O163uqLga294d7vjrfKPPVL26ukh9dFDamSFrqFA7z2H3tH6Bm9cfBBufID/a9f/xOuYOnSdXYTqyllIbByHxV0RNtXLeTd6SI30kPoc10MG+jAq9JL0fWhwTULkRyX4UfoMYONhtNd74OjRlYh4Nh7lOMceKXv19JD66CE1skLXWqD3NuuazyJsdzsNH/cALEnNA4q34vULNxw/lQ3Prk/wUvlHCCGE2AIG+hAr9Jd3HqHddTp8DK34Qf7/wPV1J4GKBag4H4SSW+EIafdkj5S9enpIffSQGlmhazHQzefQy72i8cLlU4Q7ncP39/8tWlcdRtdjd+zalYi1D/8L617H2Fwv593oITXSQ+pzXA8Z6EOs0E8uyEe5aw6W/MdK/F3WVOBIBt4+d1O/T35+83ic67T95Wrs1dNDaqSH1McKnYE+iEA3N+ub2XnYa9wJt4BP8IcVa4C6SDy8NxNVXx+CR+dUvFL+EUIIIbaCgT7ECr1s5mbsddmJvyv4ITauOAtcm4eSA2E49SgNIa+N7JGyV08PqY8eUiMrdK0Guvkc+gn3Tdjnugv/26Hv42b6NbTfn4E9exKxvskbWS/D7KKX8270kBrpIfU5rocM9CFW6OVe63HIYwf+R84EvMk8jo5Xbti/fxsiXvwXStt2sEfKXj09pD56SI2s0AdLdnZ2n/dD701jYyOioqLg7OysNnnc0NBgVaCbm3XBcy1KPHfgn5fFAzs/R0e7K6oulcOjcwpau3nvc0IIIbZFt4EuZ5OHhISgpaVFbaGhoeq6vvD19UVeXh46OzvVlpubCz8/vyFX6JXun2G/9w7MTDkKHNmExw+9ceTBFoS0/pE9Uvbq6SH10UNqZIU+WCTMq6ure5blcXBwcJ/bjx8//jvrJkyYYFWgm8+hV7tnoGjadhSvqUf3sSRcvjQTG5tDsebZdLt5wnk3ekiN9JD6HNdD3Qa6DJt3d3f3LMtjWdcX0dHRalUu20nLyclR1w21Qv/aPR2F03bg3qabeHs6BOfL5yPi1Sc41rySPVL26ukh9dFDamSFPljGjRv3nXVOTk59bt/U1ARXV1f1edLk8bNnz6wKdHOzLrunocBnJzqyS9F2yRdnzqfCvXMKml9fAyGEEGJrHKZCDw8PVyt08zn0sDDLl5dVVVWpYZ6fn//eepmHl2ES6VlddVuO/Gm70J2zF68eeiGnNgkBr/7Q83tTD8yWy2VlZXb9+3rXJz+Lioqob5jL8jpT3/CWTY36eDy0ZnlMzaHLur6wVL33V9FbqtDN59C/cUvGDp9dQMEX6HjrgrjGWeola/aE8270kBrpIfU5roe6P8u9ubnZ4lnuvYfk5Yx2mTc3n0MfzlnuN1yXYvesQiB3I169dMeypmnY8tDFrp5w3o0eUiM9pD7H9VDX16HLted9XYfeO9Dr6+vVIXY5s12aPJZ11gS6uVm1LrEoCt6HrrwluHNnOha8csXRZ8kghBBC7AG/KW6IFfptl2gUh5egtTAAV6+GYdabKbjWsoc9Uvbq6SE9pIfUyApd64FuPod+12URji8+jGel/vj60jK4dn2Cl2/u2FUv593oITXSQ+pzXA8Z6EOs0O8bI1GWcAJNN31x5sZaTH8zWVnbzR4pe/X0kB7SQ2pkha71QDc3q8EYgWMxh/DyuRcO3s/CvBeTQAghhNgLBvoQK/RGl3k4GV+Cjg4jCp4kI/HZVPZI2aunh/SQHlIjK3Q9BLr5HPpjQwiOJm/H06deWNcyFxuezrS7Xs670UNqpIfU57geMtCHWKE/NQbhZOZG3L0bhPgXHtj3LJo9Uvbq6SE9pIfUyApdD4FublazMQDnd6zCrVsxCGn9MypatoEQQgixFwz0IVboLwwz8fXBJNTVbYR3x5/x6HUNe6Ts1dNDekgPqZEVuh4C3XwO/ZVhOmrLF6Du4QG4d/4Z3d0ddtfLeTd6SI30kPoc10MG+hAr9DcGLzysDUbNi30IbP0vTehlr54eUiM9pD5W6Az0QQS6uVkdBnc0PQjAkZdrsbjlYxBCCCH2hIE+xAq922BE/a0QbHseidXPprFHyl499dFDekiNrND1EuimOfTurm410O9dD0Hq82nIb47QhF7Ou9FDaqSH1Oe4HjLQh1Cht758g7cGdzy4GoKol1NxsmUte6Ts1VMfPaSH1MgKfahkZ2f3eT/03sj90Xu3SZMmWRXoJrMeNj7Ha4MXas9Gwu/NH1HbdhKEEEKIPdFtoBcXFyMkJAQtLS1qCw0NVdcNljNnzgzYCeirQr9Z+xivDD6oLYuCa9dUtHe9YI+UvXrqo4f0kBpZoQ8FCfPq6uqeZXkcHBw86Of7+/vj2bNnVgW6aQ79YkUdWowzcfFiBGYoFbpW4LwbPaRGekh9juuhbgPd2dkZ3d3/ff9xeSzrBkN5eTlWrVo14HZ9Vegnj13GM8MsHL4SjPkvtXPJGnv19JAa6SH1sULXXaDLHHhvnJycBvXcwMBA3L171+pAN5l1sOg8moyzsbPWH8ueu4MQQgixNw5XocvQfHR0/3dGq6qqUsM8Pz//vfW+vr7qMMnONQfQ6BKMNfXeWFHn1dPTkp/ye3stl5WV2fXv612f/CwqKqK+YS7L60x9w1s2Nerj8dCa5TE1hy7rBkLm2a9duzaov9HXHHrh6kN44BKKuCduONCSppneGefd6CE10kPqc1wPdX+We3Nzs8Wz3C0NyV+6dMmqE+f6mkPfmVyM+67zMPeVETWt+zXzBuG8Gz2kRnpIfY7roa6vQ5fLzvq6Dt1SoEvoX7hwYciBbjIrL3YP7rrOh1fHJ3jWUQ9CCCHE3vCb4oZQoedGFaLOPRIenZ+wR8pePfXRQ3pIjazQ9Rbopjn03PDduOmxAHNeaSvQOe9GD6mRHlKf43rIQB9KhR6Sj2+8FmLxMyN7pOzVUx89pIfUyApdb4FuMmv77Hx87bMYoS8NIIQQQrQAA30IFXr+rHzUTI/GmsYZ7JGyV0999JAeUiMrdL0FumkOvcB/J6pmxCCr0U9TejnvRg+pkR5Sn+N6yEAfQoVeOHMHKvxisfHRbPZI2aunPnpID6mRFbreAt1k1l7f7SifFYetTSEghBBCtAADfQgV+pdKoJ8NjMf2J/PZI2WvnvroIT2kRlboegt00xx6yYztKJ2TgIKnizSll/Nu9JAa6SH1Oa6HDPQhVOiHpufiROhSfPE0lj1S9uqpjx7SQ2pkha63QDeZdXT6NhydtxRfNi8DIYQQogUY6EOo0E9Mz8GhiKU41LKKPVL26qmPHtJDamSFrrdAN82hn5q2BSULEnHixTpN6eW8Gz2kRnpIfY7rIQN9CBX6GZ/N+HJxIs683MoeKXv11EcP6SE1skLXW6CbzDrnsxF7Ypbiwqt8EEIIIVpA14GenZ2Njz76SG0bN24ccPu6ujqEh4fD2dkZBoMBBw8eHFKFft5nHT6PT0R16x72SNmrpz56SA+pkRX6cCguLkZISAhaWlrUFhoaqq7ri7t378LNzQ3l5eXo7u5Gc3MzMjIyrAp00xz6RZ8s7EpKxOXWEk29QTjvRg+pkR5Sn+N6qNtAlzCvrq7uWZbHwcHBfW6fnJw8YEU+2Aq9eloG8lISceO1tt4w7NXTQ2qkh9THCl13gS7D5lJpm5DHsq4vJk2ahL1792LKlCmYMGECYmNj8eLFC6sC3WTW1z7p2LpiKepenwMhhBCiBXQb6OPGjfvOOicnp363T01NRUdHB9rb25GWloaEhASL21ZVValhnp///klvvr6+6jDJJe8V2JixFBV3vlKXTT0t+WnP5bKyMk3p0Zs++VlUVER9w1yW15n6hrdsatTH46E1yw5TocvvJMhNSLBLpW5NhW6aQ7/ivRxrsxLw+O1NTfXOOO9GD6mRHlKf43o4pubQZV1fyElzvQO9vw6ApUA3zaFf805Gxvo4PO94oKk3COfd6CE10kPqc1wPdX+Wu5ytbuks995D8nJCnAyzS6hLk+H3pKQkqwLdZNZ1r6VI2xyHtq5nIIQQQrSArq9Dl2vP+7oO3dIce25uLiZPnqwOtcfHx1t9UpypQr/pGY/k3Bh0dLezR8pePfXRQ3pIjazQtU5fc+i1njFYmr9Ec3o570YPqZEeUp/jeshAH0KFXuexCAmF2gt09urpITXSQ+pjhc5AH0Sgm8y647EAcXsXghBCCNEKDPQhVOj33D9FbLH2Ap29enpIjfSQ+lihM9AHEeimOfQH7vMQc2iB5vRy3o0eUiM9pD7H9ZCBPoQK/aHrXEQfj2SPlL166qOH9JAaWaHrMdBNZj1ynYMlpz4FIYQQohUY6EOo0JtcAxFdNp89UvbqqY8e0kNqZIWux0A3zaE/c/HHkopwzenlvBs9pEZ6SH2O66FNA33RokVjokJvcZmJJdVh7JGyV0999JAeUqNjVujydazytatXr17VZaCbzHppnI4ll0NACCGEaAWbBnpgYKAa6tJWrVql2wq9zeCF6BtB7JGyV0999JAeUqNjVuhCfX09pk6dqob6pEmT0NjYqJtAN82htxs8EH0rQHN6Oe9GD6mRHlKf43pot5Pizp07h4kTJ/ZU7KY2fvx4zVfonQZXRN+eyR4pe/XURw/pITU6boVuorq6Gn/60596QtzUnJycNBvoqlnd3eg2GBFXPwOEEEKIVrB5oD969AhGo1ENcrk3+bNnz4b8f2VnZ/d5P/Te9B4JsHS/9MFU6G2v2tFhcENiwzT2SNmrpz56SA+p0TEr9ODg4J4wzczMHJbw4uJihISEoKWlRW2hoaHquv4CfbgVusyhNz5qwRuDJ5IbvDT3BuG8Gz2kRnpIfY7roc0vW3N2dsbNmzeHLVzCXIbtTchj6TCMZqBLhX79+kO8Mvog7YEne6Ts1VMfPaSH1OiYFXp8fPyICZeOQXd3d8+yPJZ1/QW6nFUvc/ReXl7Izc21OtDFrIqy62gx+GJV/TQQQgghWkG3X/1qqeIe7Al19+/fR2RkJNatW2fx91VVVWqY5+fnv7fe19cX29buwTODHz67P13tYcmwiamnZe/lsrIyTenRmz75WVRURH3DXJbXmfqGt2xq1MfjoTXLug10ayv03rS2tqon01lTocscevH202hymY1193w11zvjvBs9pEZ6SH2O66FuA93SHLqsG81Alzn0wrWH0egShE33/DX3BuG8Gz2kRnpIfY7roW4D3XSWe3Nzs8Wz3HsPyScnJ6vfUic0NTVhyZIlA379rKU59B0rivHANRg59wJBCCGEaAVd3z5Vrj3v6zr03oFeWlqqzoHLevnqWZk/b29vt7pC3xa/B/fcQpF3T3s3Z2Gvnh5SIz2kPlbougz00cbSHPrWBYW47T4P+ffnaU4v593oITXSQ+pzXA8Z6FZW6Fvn78Ytj/n4vCGKPVL26qmPHtJDamSFrsdAF7Ny5hbgG69PsfdRLAghhBCtwEC3skLPnZOPK9MWoKQphT1S9uqpjx7SQ2pkha7HQJc59B2z8/G1z0IceZahOb2cd6OH1EgPqc9xPWSgW1mh75qVj6oZi1D6fCN7pOzVUx89pIfUyApdj4EuZn3uvxMVM5eg/EUeCCGEEK3AQLeyQt/jtxPl/tGobi1ij5S9euqjh/SQGlmh6zHQZQ59v18+zs6OweXWEs3p5bwbPaRGekh9jushA93KCr3EfztOBcXixusT7JGyV0999JAeUiMrdD0Guph1eFYujgXH4fbrchBCCCFagYFuZYV+zD8Xh0Pj8KC9hj1S9uqpjx7SQ2pkha7HQJc59JOztqIkPB5Nb29pTi/n3eghNdJD6nNcDxnoVlbop/03Y39kAp53PGCPlL166qOH9JAaWaHrMdDFrLOzNuGLBQlo62oGIYQQohV0HejZ2dl93g+9P0JDQ79zv/TBVujnZm3A7sUJ6OhuZ4+UvXrqo4f0kBpZoQ+X4uJihISEoKWlRW0S0rJuII4cOYLg4OAhBbrMoZ/3X4sdsQmafINw3o0eUiM9pD7H9VC3gS5hXl1d3bMsjyWo+6OtrQ2enp5oaGgYcoVe4f8Z8uK1Gejs1dNDaqSH1McKXXeB7uzsjO7u7p5leSzr+iMrKwuFhYXq46EEuph10T8D25LiQQghhGgJ3Qa6pUB2cnLqc/va2loEBQX1+3wTVVVVapjn5+e/t97X1xeVfiuxNTWhp4clwyamnpa9l8vKyjSlR2/65GdRURH1DXNZXmfqG96yqVEfj4fWLDtMhS5hXl9fP6hA76tClzn0Kr80bF7BOfSxqI8e0kNqpId61jem5tBlXX8VvaVmTaDLHHqNXwo2reYc+ljURw/pITXSQz3r0/1Z7s3NzRbPch8orIc6h/61bxI2ZSaAEEII0RK6vg5drj3v6zr00Qh0qdAvT0/AhnXaPCmOvXp6SI30kPpYoesy0EcbS3PoV6fHYd0mbQY6593oITXSQ+pzXA8Z6FZW6Nd8orF2Sxx7pOzVUx89pIfUyApdr4EuZl33XoSsvDgQQgghWoKBbmWFftN7AbJ2skJnr5766CE9pEZW6LoNdJlDr/WKxGe7YjWpl/Nu9JAa6SH1Oa6HDHQrK/TbHvPx2RfaDHT26ukhNdJD6mOFzkAfRKCLWfc8wvDZvhgQQgghWoKBbmWFXu8+F2u+XMIeKXv11EcP6SE1skLXa6DLHPpDtznIKFmsSb2cd6OH1EgPqc9xPWSgW1mhP3INRMahheyRsldPffSQHlIjK3S9BrqY9dRlFjKOLgAhhBCiJRjoVlbozS4zkXE8kj1S9uqpjx7SQ2pkha7XQJc59BfG6cg49akm9XLejR5SIz2kPsf1kIFuZYXeavTGZ2fD2SNlr5766CE9pEZW6HoNdDGr3eiJz86FgRBCCNESug707OzsPu+H3pvKykqEhoZiwoQJmDhxIpKSkvDkyROrK/ROgyvWXQxhj5S9euqjh/SQGlmhjwTFxcUICQlBS0uL2iSsZV1fRERE4MKFC+js7ER3dzd2796NgIAAqwJd5tC7DUZk18zR5BuE8270kBrpIfU5roe6DXQJ8+rq6p5leRwcHGzV/yHVujWBPmdOsFqh77jCCp29euqjh/SQGlmhjwjOzs5qpW1CHsu6wSDbFhYWIioqyqpAD58fiXaDBz6/HgFCCCFES+g20MeNG/eddU5OToN6nrSPP/4Y9fX1FrepqqpSwzw/P/+99a5Gb7QZvLH7ckRPD0uGTUw9LXsvl5WVaUqP3vTJz6KiIuob5rK8ztQ3vGVToz4eD61ZdtgKXcI6KCjIqgr9z390Va9DP1S3VJO9M8670UNqpIfU57gejqk5dFlnDQNV9L0D/ZM/eaDZOBPHb6dp8g3CeTd6SI30kPoc10Pdn+Xe3Nxs8Sz33kPyq1atQkNDg/q4tbVVvcxtoA5A70A3TvZCk8ssnKnPBCGEEKIldH0duoRyX9eh9w700tJS+Pr6qusnT56MlJQUtTNgTaD/8fdGNLoE4nxTDnuk7NVTHz2kh9TICl0v9A70Sf85FQ/cglHz/AtN6uW8Gz2kRnpIfY7rIQPdikCf/FsD7rnNxbWXh9kjZa+e+ughPaRGVuh6DfSPf2NAnXso6trOghBCCNESDHRrAv1XU3HTMxwN7V+zR8pePfXRQ3pIjazQ9Rrof/5gCr7xikDT21ua1Mt5N3pIjfSQ+hzXQwa6FYH+iVKhX572KZ53PGCPlL166qOH9JAaWaHrNdCNv5qCmulRaOtqBiGEEKIlGOhWBLqrEugXfRego7udPVL26qmPHtJDamSFrtdAd//wzzg/c6Fm9XLejR5SIz2kPsf1kIFuRaB7/PpjnJ21mD1S9uqpjx7SQ3rICl3Pge756z/hdGA0CCGEEK3BQLci0L1+80ecDIrRrF726ukhNdJD6mOFzkAfRKB7/2YyjgVrN9A570YPqZEeUp/jeshAtyLQp/3mv3AkNI49UvbqqY8e0kN6yApd14H+60k4FB4HQgghRGvoOtCzs7P7vB96byorKxEWFoYJEyZg4sSJQ7ofus9vJqIkIp49UvbqqY8e0kN6yAp9pCguLkZISAhaWlrUFhoaqq7ri6ioKFRUVKC7uxvt7e3IzMxUA96aQJ/+6z+gOCpBs28QzrvRQ2qkh9TnuB7qNtAlzKurq3uW5XFwcPCgny/B7uTkZF2gf/gR9i9ayh4pe/XURw/pIT1khT5SODs7q6FsHtCybrCUl5cjKCjIqkCf8aEz9kUvBSGEEKI1dBvo48aN+866gSpuE3V1dXBzc0Ntba3F31dVValhnp+f/95631/9fyiKS+zpWclPGTbRynJZWZmm9OhNn/wsKiqivmEuy+tMfcNbNjXq4/HQmmWHq9Bramrg6uqKy5cvD7ht7wrd9wMnfLE0SbO9M8670UNqpIfU57gejqk5dFnXH6WlpTAajbh58+ag/kbvQJ/5we9RuGyZZt8gnHejh9RID6nPcT3U/VnucumZpbPcew/J79mzBwaDAXfv3h303+gd6H7//nvsTlkGQgghRGvo+jp0ufa8r+vQewe6LFtq1gS6/7+NR8EKVujs1VMfPaSH9JAVuq74bqD/L+xalaxZvZx3o4fUSA+pz3E9ZKBbEeizfjkO+RnaDXT26ukhNdJD6mOFzkAfRKAH/OJ32JmZAkIIIURrMNCtCPTZP/8ddmRpN9DZq6eH1EgPqY8VOgN9EIEe+PP/xI51nEMfq/roIT2kRnqoZ30MdCsCPehnv8X2TazQ2aunPnpID+khK3RdB3rwz36D3M2cQyeEEKI9GOhWBHrIT3+NbVtZobNXT330kB7SQ1boug70uT/9ENtyOIc+VvXRQ3pIjfRQz/oY6FYEethPfoWt2/hNcezVUx89pIf0kBW6rgN93k8+QE5eEgghhBCtwUC3ItDD/+XfsTUvkT1S9uqpjx7SQ3rICl3PgR7x418iJ1+7gc55N3pIjfSQ+hzXQwa6FYEe+eNfYOtOVujs1VMfPaSH9JAVuq4DPerHP8e2XUtBCCGEaA1dB3p2dnaf90PvzWDvgd5foC/88c+wo5B3W2OvnvroIT2kh6zQR4zi4mKEhISgpaVFbaGhoeq6gRhOoC/60U9RuD9Ns28QzrvRQ2qkh9TnuB7qNtAlzKurq3uW5XFwcPCoBvqSH/0LSk5ksUfKXj310UN6SA9ZoY8Uzs7O6O7u7lmWx7JuNAM95kc/xrmLO0AIIYRoDd0GuqVgdnJyGpFAr6qqUsM8Pz//vfXRSqCXHF3b07OSnzJsopXlsrIyTenRmz75WVRURH3DXJbXmfqGt2xq1MfjoTXLrNCtqNCX/OgnePi8UrO9M8670UNqpIfU57gejqk5dFk3moG+6Mc/w7M3tzX7BuG8Gz2kRnpIfY7roe7Pcm9ubrZ4lntfwT2cQF/w43/F667nIIQQQrSGrq9Dl2vP+7oOvXdwm1+HPtjr0S19sUxn91v2SNmrpz56SA/pISt0PdE70D/98S81rZfzbvSQGukh9Tmuhwx0KwJ9/r/8G3uk7NVTHz2kh/SQFbreA11un0oIIYRoEQa6FYEe9pMP2CNlr5766CE9pIes0PUe6HN/8qGm9XLejR5SIz2kPsf1kIFuRaCH/PTX7JGyV0999JAe0kNW6HoP9DkaD3RCCCGOCwPdikAP+tf/YI+UvXrqo4f0kB6yQtd7oM/WeKBz3o0eUiM9pD7H9ZCBbkWgB/z8t+yRsldPffSQHtJDVuh6D/RZv/hPEEIIIVqEgW5FoPv/8nfskbJXT330kB7SQ1boeg90v3/TdqBz3o0eUiM9pD7H9ZCBbkWgz/z3/5c9UvbqqY8e0kN6yApd74E+44NxIIQQQrSIrgM9Ozu7z/uhj8T2vQN9+q/+F3uk7NVTHz2kh/SQFfpIUlxcjJCQELS0tKgtNDRUXTdS21sKdJ8PtR3onHejh9RID6nPcT3UbaBLOFdXV/csy+Pg4OAR295SoE/7zXj2SNmrpz56SA/pISv0kcTZ2Rnd3d09y/JY1o3U9pYD/fcghBBCtIhuA33cuO+eoObk5DRi21sKdO//0Hags1dPD6mRHlIfK3RW6GZUVVWpYb5///731s+fP181jI2NjY2NTWtNzg3TZaBbmhOXdSO1PSGEEDKW0dxZ7s3NzRbPWu89xD7Q9oQQQggD3U7IteR9XVduac68v+0JIYQQBrqDsHfvXnVuXastPz+f+ughNdJDeuigHppPKzPQB6D3We/UN7b00UN6SI300JH0OXSgW9v7oT596aOH9JAa6aEj6XPoQCeEEELGCgx0QgghhIFOCCGEEAa6nbD2tqujiVyOZ2pa1VtZWYmwsDBMmDABEydOREpKinr9v1Y0ij75HgKTvqSkJDx58kSzr7lotfR621Oj+fuwr/ejFjysq6tDeHi4+q2QBoMBBw8e1LSHkyZN0pSHjY2NiIqKUv2TJo8bGho0o/HFixdISEhQP8uTJ09GTk6O3d+HI3GMtpVmhwv0odx21VYHA63qlQ99RUWF+vW67e3tyMzMVANeKxojIiJw4cIFdHZ2qhp3796NgIAATb7mR44cUe8K2NcXJdlLY38HK614ePfuXbi5uaG8vFx9naVTmZGRodnP9pkzZ947eGtBn6+vL/Ly8tTPirTc3Fz4+flpRqN0xlNTU9HR0aEea9LS0nDgwAFN6BvqMdqWmh0u0Idy21V7vlm0qFcOpuY3wtGiRunha01fW1sbPD091Yqo9+ttb40DBboWPExOTv5ORa7lz4q/vz+ePXumKX3jx4/X9GdFtEiQm5DH5sWDPfUN9RhtS80OF+hDue2qPd8sWtQrFVJQUJAmNcrfLiwsVEcVtKYvKytL1Wbp9ba3RtPwsHTUvLy81MpNa+9D0SdfBjVlyhT1wB8bG6sO0WrxfSifkVWrVmnOw+joaPW1lb8tTYa0ZZ1WNPYOdKnUzf++PfUN9RhtS80OF+hDue2qPd8sWtMrc5gy7FlbW6s5jaZ5ro8//hj19fWa0id+mXeCemvS0ut8//59REZGYt26dZrSJxp6D8fKfKsWPQwMDFSnCLT2Gjc1NcHV1bXnsyKPzUcR7K0xMTFRfV3l9TW9xuZ/3576hnqMtqVmVuis0AdNTU2NegC4fPmyZjXK35avYNTaCILo6a+TobX3ZWtrq3oCj5b0yd/rXb2ZDxdrxUMZUjWverWkT04olArdfA7dfEjb3hplxCU+Pl59XWUkZs+ePe+dWMgKnYH+Hlq97ao18zP20FtaWgqj0YibN2/qwtOB5vhtrc/SGdDmr7nWPOwd6FrQJycT9TccqxUPZX702rVrmvycWKoMtfZZMUfOmZAT5bSgb6jHaFtqdtiz3LV229WBzqC0p17pJcslQr2HELWiUeYqTZfeSBDJmcXmHxgtvuZaux2wnHBmGkGQYdklS5a8NwesBQ/l4G4+HCvD7+YHey1ovHTpUp8nPGlBn5zRLvPm5nPols5yt5fG9PR09W+LNjkPQU4iNR/Zsqe+oR6jbanZIa9D19JtVwdz/a+99Q5UXdpbo4weyOU4okmuXbV0nbzWbrWrtdfZ3MOpU6eq8+fm1bBWPJQhYnmNZUhWhmbNT4rTgkY5WMsllFo99kg4mr5TQpo8Ng9Me2uUTpvppEeZHpBzduzt4Ugco22lmd8URwghhIwBGOiEEEIIA50QQgghDHRCCCGEMNAJIYQQwkAnhBBCGOiEEEIIYaATQgghhIFOCCGEEAY6IWTYyNeuyrdmmd+q1sTChQvV35l/NSshhIFOCNEgbW1t6g17JLjlTnwm5G58sk5+J9+tTwhhoBNCNI4EuYS3v79/z7qAgIDvhDwhhIFOCNE4ciMXCfDjx4+rTR5nZWXRGEIY6IQQPdHZ2YkZM2aot7p0c3ODj4+Puo4QwkAnhOgIuYWp3PIyMDBQbR9//LF632dCCAOdEKIjEhMTe+bMTSfEJSQk0BhCGOiEEL1w5swZNcAjIiJ61sljWXfq1CkaRAgDnRCidUxD7RLetbW1Pevlsazj0DshDHRCiA4wDbVbGl6XdRx6J4SBTgghhBAGOiGEEMJAJ4QQQggDnRBCCCEMdEIIIYQw0AkhhBAGOiGEEEIY6IQQQghhoBNCCCGEgU4IIYQw0AkhhBDCQCeEEEKIJgL9e9/7HhsbGxsbG5tO23uBTgghhBAdVuYMdEIIIYSBTgghhBAGOiGEEELGVKCPGzfOoYzn/nJ/ub/cX+4v99fhA32w22ZnZ+Ojjz5S28aNG3X7hhnMtrKNqen9AzKYbSsrKxEWFoYJEyZg4sSJSElJQXNz85je39DQ0J79TUpKwpMnT8bs/poj+6219/Vofn61+Dkejde3rq4O4eHhcHZ2hsFgwMGDBx3q9Z00aRIDfbDbFhcXIyQkBC0tLWqTg4KsG+sHQEcJ9KioKFRUVKC7uxvt7e3IzMxUA36s7m9ERAQuXLiAzs5OdZ93796NgICAMf9+PnLkCIKDgx0i0MdKxTqYbe/evQs3NzeUl5er72fpjGdkZDhEB1U4c+bMqBSZNgt02emCggJ8/PHHapWRnJysHogtmSLrV61a1VNdy2PTtoPtxUqYV1dX9yzLYzkw2PINYcv9tfeBwV77a0IOCk5OTg6zv4L83bG8v21tbfD09ERDQ4PN39e23l97B7qt91f+f3tW5Pb+/Pr7++PZs2f6DvQlS5agtbVVbfJ4/fr1Fg1ct24dFi1a9N62ss6aN78M48hB3vyAL+ts+Yax5f5qIdDtsb8mpKcfFBTkEPsr7+XCwkJ1lGIs729WVpa6n/Z4X9t6f01DsNIp9fLyQm5u7pjeX9nXvXv3YsqUKWqgxsbG4sWLFw7x+ZVjlXQKRoO+A/32beD48aE1ea4FAx89etSz3NjYqL6YlkyR9fJ7822lJzXcIaz+KrgR3l2b76+1295W/h0f4j95rlb2V5C5OBm+q62t7XePlVdriO22pl5fafL8+vp6W+2uzfdXXkvzDtpAzxnh3bXr+/n+/fuIjIx8LzS+w/PnMm49tCbP1cDxOTU1FR0dHWq1m5aWhoSEhD63f/jwIaqqqobU5Llaen0DAwPVKQfbBvr580Bm5tCaPNfKgDX/vTXbjlSFPsK7a/P9tXbb88q/zCH+k+dqZX9ramrg6uqKy5cvD7jHyqs1xKad/TW9l/Pz8/sfkRjZ3bX5/sq+mXdYBnrOCO+uXV9fQSpBGd7tk4YG4OLFoTV5rgaOz+ZD3BLs/U0hXbt2DXv27BlSk+dq5fWVqd/o6OhRG3mw6ZC7NT2i3tta2yOyNIcu62w5pGPL/dXCkLut97e0tBRGoxE3b950iP21ZsRJ7/tr6axgW7637f36DhjoOt9fOUm5d6DbekrUHq+vnMdlqYOhy0CXuQc50UVaf3MWst40vyHbyvyF+fCTvNEHOqHAdJa7nD1pj7Pcbb2/Wgh0W+6v9LzlUpfRGrrS2v7KnFvDt5WV/D9yhqytO6j2eD/b631t6/2Vk7JMIxJNTU3q/zda86xa2F85IU6G2SXUpcnwu1yKOZbfz5cuXRr1E7NtGugyTDh58mR1aMU0f2LJQHmB5RIG2U5aenr6e9vKJTuyfqAPuRz07HUduq33197XsWphf21dwdlyf2U0wtfXV91G/qatr7u3x+fX3oFur9d36tSpamCYV7Bj8fWVE/9Mfy8+Pt7mJ8XZen+lqJRLT8dMoDsS3F/uL/eX+8v95f7aEgY695f7y/3l/nJ/ub8M9MFjyy/B0ALcX+4v95f7y/3l/o7JQCeEEEIIA50QQgghDHRCCCGEgU4IIYQQBjohhBBCGOiEEEIIYaATQgghDHQbwi824P7SD/pB6Af9cLBAH8y2ckecqKgo9Q4+0uRxg4XbBo6V/ZXvQZb7CcsXJsj3E+fk5Oj2A2nt3bj6Ijs7227f5a81P+z1Hf9a9aOyshJhYWHq52XixIk2/258Lfoh3zVu8kNulPLkyROHPn6YEF/00qEYs4EuNzrIy8tDZ2en2uRGAH5+fmP2DSofQNMNBuRmAnInowMHDozZ/R1oW9Pd9uROe/a4257W/NB6pWNrP6SDX1FRod5bXj4vmZmZasA7qh8RERHqjUPkWCmeyA1HAgICHP7zcuTIEfUOaQx0C8YVFBSo95GVXqDcLtD8bkK9724jtw40VVfy2LTtYO+uNX78+O+ss+XX/dl6f+VvmP//8tiWByhb7+9AH0gJ8+rq6p5leTzaty7Ush/2DnSt+mFCQszW95fXsh9j/Xg5GD/kVqmenp7qyC4D3YJxpnvKSuvv/rNy60C556z5tub3nx2MudHR0WpVLh9UaTIELets+Qa15f72DnSp1GWqYazu70Dbyr7L625+wHZkP7QQ6Fr0w0R5eTmCgoLox7eflcLCQnUUw5H9yMrKUn3Q8sjWoAO9+fhx3I6JGVKT51oy7tGjRz3LMsc9ZcoUi8bKevm9+bbSc7PmBWtqaoKrq2tPT00e93cT+uPHmxETc3tITZ5r7/1NTExUh9kl1E1D7v1VHMebjyPmdsyQmjzX3vs70LaW1vfnR7OyT7eVfRtKa9aBH9b+X83Ke/q28t4eSmvWwOfBmm3r6urg5uaG2trafj4vyvHh9u0htePNzbp6f0iT/7++vr7P7aqqqrBly5YhNXmu1v2Q94J5B4+BbuUB1vz31mzbF+Hh4WqFbj6H3t8Q9GgEui33V06Ki4+PVyt1ecPv2bMHkyZNsmmg23J/R7pCH41A15IfWgh0LfpRU1OjdvYvX77c73ajEeha9MP0WcnPz+93xGI0Al1Lfsi+m3doOOQ+zB5Y722t7YFZqsZsPUdmy/3tzcGDB9UT5cb6/lozhy7rHNUPLQy5a82P0tJSGI1G3Lx5k34M8hjqKH70novX8hUidgt0meuQEw2k9TdHIutN8ymyrcyXmM+RyIkQ/Q2fC3JGu8ybm8+h2/Isd1vvb3p6unrZjeyrzAfKyRz9DZnpfX8H+kCaznIXT+xxlrvW/NBCoGvJDxnBMhgMuHv3Lv1QkBPLTJf1yt+Ryzxt3QHW4ueFFXo/hsgwjlwjLcPCpkusLBkmc8AZGRnqdtIkrMy3lUsqZH1/JkuYma4zlSaPbR1wttxfqcil5yrbyXSDzAva+gBly/0dTO9ZDkr2ug5da37Yu9rQgx+29ERrfshohVzqK+tFk62vy9fi8YOBPgYM4f5yf+kH/aAf9EOPMNC5v9xf+kE/6Af9YKAPHlt+SYEW4P46NvSDftAP+jFmA50QQgghDHRCCCGEMNAJIYQQBjohhBBCGOiEEEIIYaATQgghhIFOCCGEMNBtCL9YgH7QD/pBPwhxsEAfzLYDfUev1u+kY2s/BPn+d/keeLnNqNy4Qr4f3lH9sPT+6O92tGPdD7mjVVRUlPrekCaPTTfzcEQ/5HbFCQkJ6heiyPePy82fCGGgj9IHcqBttd6jt7UfctcpNzc39U5tcsc2uTGD3ADBUf3ozZkzZ2x+gxct+SE37sjLy0NnZ6facnNzbXr3Qq35IbcmNt1ARG4WkpaWhgMHDjBRiGMEunwwCgoK1PvWSq82OTlZ/SBY+uDIermdn+lOWfLYtK21VbVWA11rfsjft2dFrjU/euPv7z/oWzKORT/Gjx//nXW2/LpOrfkhGsz/vjyWOzoS4jCBbrqHrbT+7ncr97aVe9yab2t+v9uRqtBlCNXJyQleXl5qxWHrANOSH+LF3r17e27BGhsbqw4rOqof5siohYSCI78/oqOj1c+IjN5IkyFmWeeofvQOdKnUZSqCEG0GelUVsGXL0Jo818IH49GjRz3LMicn4WHpgyPr5ffm20rPfLQO2Pfv30dkZOR7H/rejLAdmvND1vceQpQ5wj79UP5tGeI/ea6e3h+BgYHqlET/yD5tGWLTvh9NTU1wdXXtqWzlcb8jFiNrh+b8SExMVD8j8lkxfV6kOCDEYQK9N+YfAPPfW7PtSBywBenJy/CcLQNdS35IddG74uhvSHU0Al2L74/q6upBVqIjH+ha8kNOlpQK3XwOvd8h5lEIdC35IaNX8fHx6mdEOhB79uzR1EmThIE+6kPu1vSwe287mhXYYAJ9rPsRGhpq1yFErb4/goODce3aNZt/MLXmh6Xq05YVqdaPH3L+iZwoR4jDBLrMZbW1tamtvzkwWW+aL5NtZT7MfDhcgnewJyj1dxJYfX29+liGE+Xv2XKeVGt+yAHJfAhRht9teYDSmh/CpUuX1EC3B1rzQ85ol3lz8zl0W57lrjU/0tPT1StBxAs5x8LT07PneEKIQwR6fn6+es2mDFOZ5mstfXAkUOSSKdlOmnx4zLfdvXu3ur6/g/FA15mXlpaql+LI+qlTp6ofePMK1RYHKC35IcgwqkmPDCfa+qQ4rfkhoxYXLlywW6BryQ8JKxliN/0NeWzLANOaH9IBNp1AKtMR8h0OhDhUoBP6QT/oB/0ghIHOAxT9oB/0gxBi70C35ZdQ6AH6QT/oB/0gRJeBTgghhBAGOiGEEEIY6IQQQoiDBTobGxsbGxubPltPoBNCCCFE3/z/KnEs3y+9L1cAAAAASUVORK5CYII=",
      "text/plain": [
       "BufferedImage@48e8b199: type = 2 DirectColorModel: rmask=ff0000 gmask=ff00 bmask=ff amask=ff000000 IntegerInterleavedRaster: width = 500 height = 400 #Bands = 4 xOff = 0 yOff = 0 dataOffset[0] 0"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plot(rocs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "data": {
      "text/plain": [
       "0.97689,0.97628,0.97272,0.97096,0.97659,0.97679,0.97456,0.97253,0.97606,0.97698,0.97540,0.97161,0.97517,0.97693,0.97605,0.97278,0.97390,0.97665,0.97632,0.97352"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "val aucs = mean(rocs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The maxi2 function will find the max value and its index."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "data": {
      "text/plain": [
       "9"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "val (bestv, besti) = maxi2(aucs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And using the best index we can find the optimal parameters:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "data": {
      "text/plain": [
       "0.50000,0.10000"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "texpparams(besti) \\ lrateparams(besti)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Write the optimal values in the cell below:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Note:</b> although our parameters lay in a square grid, we could have enumerated any sequence of pairs, and we could have searched over more parameters. The learner infrastructure supports more intelligent model optimization (e.g. Bayesian methods). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "name": "scala",
   "version": "2.11.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
